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It is well-known that helical surface states of a three-dimensional topological insulator (TI) do not 
respond to a static in-plane magnetic field. Formally this occurs because the in-plane magnetic field 
appears as a vector potential in the Dirac Hamiltonian of the surface states and can thus be removed 
by a gauge transformation of the surface electron wavefunctions. Here we show that when the top 
and bottom surfaces of a thin film of TI are hybridized and the Fermi level is in the hybridization 
gap, a nonzero diamagnetic response appears. Moreover, a quantum phase transition occurs at a 
finite critical value of the parallel field from an insulator with a diamagnetic response to a semimetal 
with a vanishing response to the parallel field. 



oo 
(N 



CD 



O 
> 

o 

O 



X 



I. INTRODUCTION 

Topological insulators (TI) are long-knovifn and rel- 
atively common materials, which have recently been 
shown to exhibit phenomena, closely related to the quan- 
tum Hall effect (QHE), but under significantly less strin- 
gent conditions, requiring neither the two-dimensionality, 
nor the high magnetic fields or the ultralow temperatures, 
necessary to observe the QHE.^ In particular, TI materi- 
als, under the condition of unbroken time-reversal sym- 
metry (TRS), have robust metalHc edge states. These 
edge states are not chiral, like in the QHE case, but 
helical, which means that for each momentum on the 
Fermi surface of the edge state, the spin has a rigidly 
defined direction, transverse to the momentumi^i^ The 
spin direction thus rotates by 27r around the Fermi sur- 
face, which can be thought of as a surface manifestation 
of the nontrivial topology of the bulk bandstructure of 
the material. This rigid spin-momentum coupling is nat- 
urally viewed as a very interesting property of the TI 
surface states from the point of view of spintronic ap- 
plications.''^- Generation of current with 100% spin po- 
larization (giant inverse spin-galvanic effect)S-iI and novel 
magnetoresistance effectsS. are some of the concrete pro- 
posals that have been discussed in theoretical literature. 
One practical obstacle that currently stands in the way of 
realizing these proposals experimentally, is the large bulk 
conductivity that most known TI materials possess due 
to unavoidable imperfections in the crystal structure and 
composition. While different methods of dealing with 
this problem have been proposed, perhaps the most at- 
tractive one is to grow TI samples in the form of ultrathin 
films, so that the bulk conductivity contribution, relative 
to that of the surface states, is naturally reduced and does 
not mask the surface transport effectsJ^ 

Apart from resolving a practical problem, ultrathin TI 
films have interesting physical properties of their own, 
that bulk samples do not have. Nontrivial collective ef- 
fects, such as excitonic superfiuidity,^^ have been theoret- 
ically proposed to occur in ultrathin TI films. Magneto- 
optical response of thin TI films has been predicted to 
possess universal properties, reflecting topological order 



of the bulk materialJ^ Yet another group of interesting 
phenomena in TI films occur when the thickness of the 
film becomes comparable to the "penetration depth" of 
the helical surface states into the bulk and the top and 
bottom surfaces thus start to hybridize. This typically 
happens at a thickness of 5 to 10 quintuple layers (QL), 
i.e. of the order of 10 nm.^*' In particular, if the Fermi 
level is situated within the hybridization gap, the film 
may (depending on the sign of the hybridization matrix 
element) enter the two-dimensional quantum spin Hall 
insulator phase,-i2, or the quantum anomalous Hall insu- 
lator phase if TRS is brokenjii 

In this paper we show that a thin TI film with hy- 
bridization between the helical metal states on opposite 
surfaces possesses a highly nontrivial response to mag- 
netic field, applied in the plane of the film. It is well- 
known that TI surface states respond strongly to a per- 
pendicular magnetic field: the gaplessness of the surface 
Dirac spectrum is guaranteed by TRS, but once TRS is 
broken by the perpendicular field, a gap opens up at the 
Dirac point. If the Fermi level is in the gap, the surface 
becomes insulating and exhibits half-integer-quantized 
Hall effectiii. Response to parallel field, however, is com- 
pletely different. While it is known that an isolated he- 
lical surface state does not respond to the parallel field, 
apart from a rigid shift of the Dirac dispersion in mo- 
mentum space, this changes, as we demonstrate below, 
when the surface states on the opposite surfaces of the 
film are hybridized. We demonstrate that in this case the 
film undergoes a quantum phase transition, driven by the 
parallel field, from an insulating to a semimetallic state. 
The two states can be distinguished by their magnetic re- 
sponse: while the insulator has a diamagnetic response, 
the response in the semimetal is identically zero (this 
does not include the magnetic response of the bulk TI 
material, see below). We note, in passing, that parallel 
field response of the helical edge states has also been dis- 
cussed for two-dimensional TI samples in the form of a 
thin strip. While there are some similarities, the phase 
transition we discuss below is absent in this case (it is 
replaced by a symmetry-related band crossing, with no 
nonanalytic behavior of either the band dispersion or the 
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magnetic susceptibility) . 

II. FIELD-DRIVEN INSULATOR TO 
SEMIMETAL PHASE TRANSITION 

We consider a thin TI film, subject to an in-plane mag- 
netic field of magnitude B. We assume for concreteness 
that the field is directed along the x-axis, with the z-axis 
normal to the plane of the film. Choosing Landau gauge 
for the vector potential A — —yBz and assuming that 
z — ±d/2 for the top and bottom surfaces of the film 
correspondingly, we can write down the Hamiltonian for 
this system in the following form: 

H^Y. t^^'^' {zxa)-{k- KBT^y) + Ar^] c^c^. (1) 

k 

Here vp is the Fermi velocity, characterizing the sur- 
face Dirac dispersion, A is the tunneling matrix ele- 
ment between the top and bottom surfaces"'^^, kb = 
djlP' — 1 /2mvFp is the magnetic wavevector, which con- 
tains contributions from the Aharonov-Bohm phase gra- 
dient (first term) and the Zeeman coupling (second term) 
and I — yj c/eB is the magnetic length (we will be using 
h = 1 units). For film thicknesses greater than 1 QL, 
d > l/mvp ~ 1 nm, and we will neglect the Zeeman 
contribution to kb henceforth. We have introduced two 
sets of PauH matrices, cr and r, acting on the spin and 
the top and bottom surface pseudospin degrees of free- 
dom correspondingly. The spin and pseudospin indices 
are suppressed for brevity. 

It is well-known that in-plane magnetic field has no ef- 
fect on helical surface electrons, since the corresponding 
term in the Hamiltonian can be eliminated by a gauge 
transformation of the electron field operators ^'(r) 
^'(r)e''^ kbw^ which simply shifts the surface Dirac cones 
by ±hb along the y-axis in momentum space. However, 
since the corresponding phase factors are complex conju- 
gates of each other for the opposite surfaces of the film, 
the tunneling term in Eq.([l]) breaks this "gauge symme- 
try" (this is known as chiral symmetry in particle physics) 
and thus leads to a nontrivial dependence of the energy 
of the system on the parallel field. 

To see this explicitly, we diagonalize the Hamiltonian 
Eq.([T]), obtaining the following surface band dispersion: 



ek± = y + A2 + e| ± 2es ^Jt'lk^ + A^, (2) 

where sb = vf^b is the "magnetic energy" and the sec- 
ond pair of bands is given by — ek±. The most inter- 
esting feature of this bandstructure is a quantum phase 
transition, that occurs as a function of es/A. This tran- 
sition is illustrated in Fig. [TJ Indeed, when < A 
and focusing on the positive energy part of the spectrum, 
the bandstructure consists of a pair of quadratically dis- 
persing bands, split by the TRS-breaking gap of magni- 
tude 2eB at k = 0. The bottom of the lower band is 



above the Fermi energy ep = hy A — €b and the film 
is thus an insulator with an energy gap of 2(A — e^). 
When e_B = A, the gap closes and the system becomes 
a semimetal. Right at the transition point, the conduc- 
tion and the valence bands touch at a single point, which 
has an unusual character: the band dispersion away from 
this point is linear in the x-direction (i.e. along the mag- 
netic field) and quadratic in the y-direction. Such "semi- 
Dirac" band-touching points also occur (via a different 
mechanism) in grapheneJ^ and have interesting proper- 
ties. Here we will, however, focus on the physics of the 
field-driven semimetal-insulator transition rather than on 
the properties of the critical point itself. When es > A, 
the film is semimetallic and the low-energy part of the 
dispersion consists of two Dirac points, separated by a 
wavevector 2ko = {2/vf)\/ f^B ~ along the y-axis in 
momentum space. 

To understand the nature of this phase transition bet- 
ter, it is useful to approach it from the opposite limit, 
namely the limit of zero hybridization A = 0. In this 
limit, the bandstructure consists of two Dirac cones, 
shifted by 2kb along the y-axis, as illustrated in Fig. [1] 
Focusing on the fc^; = section of the band dispersion, 
shown in Fig. [U there are four band-crossing points: two 
corresponding to the Dirac points at ky = ±ks and two 
at ky = 0. Turning on a small intersurface tunneling ma- 
trix element A, one would expect, naively, hybridization 
gaps to open up at all four crossing points. This, how- 
ever, is not what happens. Gaps of magnitude 2A in fact 
open up only at the two crossing points at ky = 0, hut 
not at the Dirac points. The reason for this is topolog- 
ical, as was noted in a different context by Klinkhamer 
and Volovik.— Each Dirac point can be characterized by 
a topological invariant: the spin winding number around 
any surface in momentum space, enclosing the point (the 
winding numbers have opposite signs for the positive 
and negative energy bands, emanating from each Dirac 
point). The winding numbers are equal to unity by mag- 
nitude and have opposite signs for the two Dirac points. 
As long as the Dirac points are separated in momentum 
space, they are stable with respect to the intersurface 
hybridization, since only states in momentum space with 
the same spin can be hybridized. Thus the Dirac points 
can not be eliminated by turning on a small tunneling 
matrix element in the Hamiltonian. The pair of Dirac 
points can only be annihilated (like a vortex-antivortex 
pair) by merging the points together, which is precisely 
what happens when = A. The parallel- field driven 
semimetal-insulator transition in a thin TI film is thus a 
topological transition, in the sense that the two sides of 
the transition are distinguished by the presence or ab- 
sence of separated neutral pairs of topological defects 
in momentum space, i.e. the Dirac points. An impor- 
tant caveat one needs to keep in mind is that once an 
in-plane magnetic field is applied to the film, the Dirac 
points are no longer protected against disorder. Even 
though the in-plane field does not open a gap, it moves 
the Dirac cones to different points in momentum space, 
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FIG. 1: (Color online) fci = section of the surface band 
dispersion, plotted at different values of the tunneling ma- 
trix element A and fixed nonzero magnetic field, (a) A = 
bandstructure: two Dirac cones shifted by 2kb with respect 
to each other along the y-axis. (b) Bandstructure for A < es, 
i.e. in the semimetal phase. Hybridization gaps open only at 
the two ky = crossing points, (c) Critical point A = es. 
The dispersion of the two bands that touch is quadratic along 
the 3/-direction, but remains linear in the x-direction. (d) 
Bandstructure in the insulating phase, A > es. 



as discussed above. Any translational symmetry break- 
ing perturbation can then hybridize the Dirac cones, if it 
has a Fourier component at the wavevector, connecting 
them. Thus, strictly speaking, Dirac points continue to 
be topologically protected only if translational symmetry 
is imposed. However, we expect that our results will not 
change significantly as long as the disorder scattering is 
weak, namely as long as At 3> 1, where t is the mean 
scattering time. 

In the remaining part of the paper we will demonstrate 
that the two phases, the semimetal and the insulator, can 
be sharply distinguished by their magnetic response to 
the in-plane field: while the insulator has a nonvanish- 
ing diamagnetic response, the response of the semimetal 
is identically zero. The magnetic susceptibility x can 
thus be used as an "order parameter" , characterizing the 
above phase transition. 

Magnetic susceptibility can be calculated as the second 
derivative of the free energy of the system: 



In 1 1 -h e 
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(3) 



with respect to the magnetic field x{B) — 
— {l/dL'^)d^F/dB'^, where L is the linear dimen- 



sion of the film in the a;, y-directions. A straightfor- 
ward calculation assuming T — and ep ~ gives 
X{B) = {vFdel2cfl{B), where I{B) is the following 
momentum-space integral, which includes the contri- 
bution of all filled (i.e. negative-energy since we take 
EF = 0) single-particle states: 
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(4) 



The last two terms in the square brackets in Eq.(|3]) have 
been added by hand to remove the ultraviolet divergence 
in X, which is a known feature of Dirac fermions. The 
subtracted terms correspond to the magnetic susceptibil- 
ity calculated at A = 0, which is equal to zero from the 
discussion above. 

The two-dimensional momentum integral in Eq.(j4]) ap- 
pears to be rather formidable. Quite unexpectedly, how- 
ever, it can be evaluated exactly and gives a very simple 
final answer. Indeed, integrating over kx first, we obtain 
a significantly simplified expression: 
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I{B) = —^ dky In 
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(5) 



The integral in Eq.([5l) behaves very differently depend- 
ing on the presence or absence of Dirac points in the 
spectrum, i.e. in the insulator and semimetal phases. 
In the semimetal the wavevector ko = (I/wf)^/^^ ~ 
which determines the location of Dirac points on the y- 
axis, is real. The integrand in Eq.([S|) then has logarith- 
mic singularities that all lie on the real axis. As a re- 
sult, the integral is easily shown to evaluate to zero. In 
contrast, as one enters the insulating phase, the Dirac 
points merge and the insulating gap opens up. The 
wavevector now becomes imaginary and two of the 
logarithmic singularities in Eq.Q move to the imagi- 
nary axis. The integral is then nonzero and is given by 
I(i?) = — (l/7rw|.)\/A2 — e^, i.e. it is proportional to 
the separation between the two logarithmic singularities 
of the integrand in Eq.([5]) on the imaginary axis. Thus 
we obtain the following result for the magnetic suscepti- 
bility: 



X(S) = -e(A-eB 



a^maod 
47r 



A2 



(6) 



Here Q{x) is the step function, a = j c is the fine struc- 
ture constant and oq — l/rae^ is the Bohr radius. Sus- 
ceptibility x{B) is diamagnetic in the insulating phase 
and is identically zero in the semimetal. The diamag- 
netic response of the insulator arises due to circulating 
persistent currents, which fiow in the opposite directions 
on the top and bottom surfaces of the film to partially 
screen the applied in-plane field. Note once again, that, 
as directly follows from the properties of Eq. ([S]) , whether 
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the magnetic susceptibility is zero or not, depends only 
on the presence or absence of Dirac points in the spec- 
trum, i.e. on its topological properties. 

Integrating xi^), we can also obtain the magnetization 
density as a function of the field: 



M{B) = 
2'kvf 



A 



A2 



arctan 



VA^ 



(7) 



where rj — 0+. The semimetal is thus characterized by a 
constant, field- independent magnetization, which is pro- 
portional to the square of the hybridization matrix ele- 
ment. We would like to point out that the above results 
are valid as long as the surface state dispersion can be 
well approximated by the linear Dirac dispersion, i.e. as 
long as the relevant energy scales A and eg are signifi- 
cantly smaller than the bulk band gap. 



III. DISCUSSION AND CONCLUSIONS 

Let us now comment on the experimental observabil- 
ity of the proposed phase transition and the nonanalytic 
behavior of the magnetic susceptibility x{B) which ac- 
companies it. First, let us estimate the field strengths 
that would be necessary to observe the transition. We 
take the tunneling gap to be A '--^ 1 meV and d ~ 10 nm. 
Using vp ^ 10^ cm/s, we obtain the critical magnetic 
field to be of the order of 1 Tesla. The 1 meV hybridiza- 
tion gap also requires correspondingly low, i.e. ^ 10 K 
temperatures. The above conditions on the magnitude of 
the magnetic field and temperature are relatively easily 
realizable experimentally. Another important issue is the 
magnitude of the magnetic response under these condi- 
tions. Assuming as above that A ^ 1 meV, we obtain 
X ~ 10~^ at B = 0. Thus x under the above conditions 
is approximately one to two orders of magnitude smaller 
than typical electronic magnetic susceptibilities of com- 
mon metals. Measuring susceptibility of this magnitude 
is by itself straightforward, however one needs to be able 
to extract it from the total signal, which will contain 



a significantly larger diamagnetic contribution from the 
bulk TI material. It should be possible to do it by differ- 
entiating the measured susceptibility with respect to B, 
since the bulk atomic diamagnetic susceptibility can be 
expected to be constant for fields of magnitude of a few 
Tesla. Probably the most difficult experimental issue is 
how to position the Fermi level at the Dirac points with 
high precision, since as-grown films will generally turn 
out to be doped away from charge-neutralityJ^ It should 
be possible to solve this problem by gating the film. 

In conclusion, we have shown that an ultrathin film 
of a TI material with hybridization between the top and 
bottom surfaces exhibits a nontrivial response to in-plane 
magnetic field. Namely, the film undergoes a quantum 
phase transition from an insulator to a semimetal at a 
finite critical value of the field. Coming from the large 
field (semimetal) side of the phase diagram, the transi- 
tion can be described as annihilation of two Dirac cones, 
characterized by equal in magnitude but opposite in sign 
spin winding numbers. We have also demonstrated that 
the insulator and the semimetal can be sharply distin- 
guished by their response to the in-plane magnetic field: 
while the insulator has a diamagnetic response, the mag- 
netic susceptibility of the semimetal is identically zero. 
To put our results in a broader context, it might be use- 
ful to compare in-plane field response of an ultrathin TI 
film and that of a bilayer two-dimensional electron gas in 
the quantum Hall regime at filling factor v = 1.— In the 
latter (different) in-plane field driven quantum 

phase transition is also observed and is understood to 
be a consequence and one of the smoking-gun features of 
collective phenomena, namely excitonic superfiuidity— 
In a TI thin film, a field-driven quantum phase transition 
arises in the absence of any collective effects, but due to 
topological order, characterizing the bulk TI material. 
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